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ABSTRACT 

The digital computer has become an important tool in the on-line 
optimal control of large systems. Because computers are discrete-time 
in nature, it is necessary to select an interval of discretization when 
controlling a continuous -time system; this introduces another parameter 
over which to optimize. 

In this thesis, we discuss how the characteristics of a particular 
computer, such as multiplication time, will determine the range of 
choice for this parameter, and how such a choice will affect system per- 
formance. We also discuss how a suboptimal control, involving less 
computation, will allow us to control the system at more frequent inter- 
vals and thus, in some cases, lead to better results than if we used the 
usual optimal control. 

We begin by making a formal statement of the optimal control prob- 
lem which includes the interval of discretization as an optimization 
parameter, and extend this to include selection of a control algorithm 
as part of the optimization procedure. We show how the performance of 
the scalar linear system depends on the discretization interval. We go 
on to develop discrete-time versions of the output feedback regulator 
and an optimal compensator, and use these results in presenting an 
example of a system for which fast partial-state^ feedback control better 
minimizes a quadratic cost than either a full-state feedback control or 
a compensator. 
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Chapter I 


Introduction 


1.1 Motivation 

The digital computer has become an essential tool in the solution 
of practical optimal control problems. Even "simple" lower-order sys- 
tems require an inordinate amount of computation (such as in the solu- 
tions of the Riccati and Kalman Filter equations in [1] and [4]) in 
order to achieve the optimal trajectory. For a large-scale system, the 
problems are even greater. In particular, it may be desirable to use 
the computer "on-line" to control the system, that is, to include the 
computer in the feedback loop itself. 

NatTirally, an arrangement of this kind gives rise to new problems 
and limitations, stemming directly from the special nature of the com- 
puter itself. For example, let us consider the deterministic- linear 
regulator problem [31, in which the system is described by the equations 

x(t) = F(t)x(t) + G(t)u(t) (1.1) 

2 .(t) = H(t)x(t) , (1.2) 

in which we wish to determine 

u(t) «£(2.(t)) (1.3) 

in order to minimize a given functional J(x(*)» t^, t^^) , where [t^, t^l 

represents some interval of time. Often, the functional form of g(*) 


- 5 - 



- 6 - 


xn (1.3) will also be restricted. In any case, the state 3c(t) of the 
system will have a constantly changing value, and consequently, so will 
the optimal control u*(t) • This cannot be reconciled with the discrete- 
time nature of the .computer; the jnachine can in no -way continuously 
monitor the output y^(t) , nor can it continuously change the input. 

The usual solution to this problem is to sample the plant output 
at a certain select interval A of time, and similarly, calculate a 
control at intervals, using an external device to hold the actual input 
constant over each period of time. This is illustrated in Figure 1. 

In a sense, what we need to do is constriuct a discrete-time 

system 


(1.4) 

^ - C(k)^ (.1.-5) 

for which the behavior of 3^ is in some sense "close" to the behavior 
of x(t) when the value of 3^ is compared directly with that of x(kA) . 

A system in which the states of (1.1) and (1.5) ar e equal at these 
points is in fact dete:rmined by Levis [10] . 

a 

It should not be surprising, though, that the selection of the 
sampling interval A will affect the performance of oxir on-line control. 
This thesis will discuss ways in which the best choice for A depends on 
the characteristics of the computer facility upon which the control is 
implemented, and how this dependence in turn affects the choice of a 


control algorithm. 
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Fig.l Inclusion of fhe Computer in the Feedback Loop 
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1*2 Qptiiaizatxon on the Di.gi.tal Computer 

When an optimal control for a discrete-^time system is deteinuined 
on-line by a computer, there is a certain computation time associated 
with the calculation of each step* Thus a certain minimal interval of 
discretization can be associated with a given control algorithm. For 
example, suppose a computer takes t seconds to perform any simple compu- 
tation (like multiplication) , and for a particular algorithm, n simple 
computations are needed to compute the optimal control. If we use A to 
symbolize the inteirval of discretization for a discrete-time model, 
then for the aforementioned algorithm, the mmimiam A that is physically 
possible IS seconds. If we denote the control algorithm 

as 1 , the possible range for A is ^ • 

For the remainder of this work, we will restrict our attention to"^- 
time- in variant" systems, and consider only steady-state controls. This 
gives us the advantage of being able to associate with each algorithm 
a very simple parameter set. For example, in such a case, the feedback 
matrix given by the familiar Riccati equation becomes an algebraic 
equation with a constant matrix solution, rather than one that is a 
function of time. 

Let us state this in a precise manner. Henceforth, for conven- 
ience , we will no longer Tinderline vectors and matrices . The reader 
should assume all upper case Greek and Roman letters to be matrices and 
all lower case letters to be vectors, except for units of time (such 
as A or t) , indices (i, k) , or where otherwise specified. 
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We will consider deterministic systems of the form 

x(t) = f(x(t), u(t) , t) (1.6) 

y (t) = h(x(t) , t) ; (1 7) 

a feedback control algorithm i will be of the form 

u(t) « K) , (1.8) 

for ^ t < ^k+l' ^ “ ^k+1 ~ K IS the set of constant para- 

meters that we are to optxmize over in the control law, and z(tj^) is 
the information that may be obtained from y(t) , for t tj^. 

Now let be an index set of control laws. We can characterize 
the usual optimization problem [10] as follows. If we are given 
1 e , and a particular A e [^mxn(i) ' ' find the K that minimizes 

a given cost functional J(i, A, K) , where K is a parameter set as 
mentioned above. 

Using this format to state the problem suggests a generalization 
that will be the crux of this thesis. In the case of the linear system 
with quadratic cost, where the control is based on a full state obser- 
vation, and IS a linear feedback law, Levis [10] has raised the question 
of how to determine the optimal interval of discretization for a given 
control algorithm. He has pointed out that for some systems, costs 
do not decrease monotonically as A decreases (see Chapter II for a 
further discussion) ; in fact, he was unable to determine the optimal A 
for the problem he considered. Let us state this question as follows: 
suppose we are able to minimize J(i, A, K) by finding a suitable K = K*, 
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as xn the problem ^ust mentxonedj sxnce thxs xs for a partxcular x and A, 

let us then say that K* = K*(x, A) . In other words, for a gxven x and 

A, there xs a "best” K, K*(x, A) . Then for a gxven x e , we wxsh to 

fxnd the A £ [A , ^ that mxnxmxzes 
inin (i; 

JU, A, K*(i, A)) e (1.9) 

Mow since A , . is certainly a function of i alone (for a 
inin(i) 

given computer}, the A that minimizes equation (1.9) is also a function 

of 1 , that IS, A = (as we shall see in Chapter II, it is not 

always true that A^ . . = A , • Then the cinicial question becomes : 

^ f(i) min(i) 

what IS the control algorithm i that minimizes the cost? In other words, 
find the i £ that minimizes 

J(i/ K*(i, (1.10) 

At this time, there does not appear to be a neat general approach to the 
solution of this problem. The purpose of this work is to provide an 
impetus and suggest a start for investigation along this line. 

1.3 The Scope of this Work 

Because this area is extremely broad, too broad, in fact, to 
develop a unified theory, we will be content in this document mainly 
to illustrate some of the points mentioned in the previous subsections. 

In Chapter II, we will discuss the selection of the optimal sampling 
rate for a full-state feedback law, and present, as an example, a 
simple class of systems for which the optimal A can be solved for 
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explicitly. In Chapters III and IV , we will derive the sampled-data 
versions of two alternative control laws, expressing each as a function 
of the sampling rate A, In particular, Chapter III will deal with 
partial-state linear feedback controls, while Chapter IV will give 
results on optimal reduced-order compensators. In Chapter V, we will 
give a specific example of a system for which, due to our greater freedom 
of choice in selecting A that results from computational considerations, 
we actually are able to decrease a system cost by choosing a **siaboptimal" 
control algorithm over an "optimal” one. 



Chapter II 


Oh Choosing the Optimal A for a Given Algorithm,* 
The Scalar Case 


For a given control algorithm, it is clear that the continuous 

controller (i.e., A -»■ 0) will minimize the system cost better than any 

particular discrete-time controller. Thus it might appear that the 

problem of selecting an optimizing A for a computer control is trivial, 

that the best choice is A •= A . As Levis [10] has shown, this may 

min 

not always be the case. For the second order system 




■ 0 l" 


■ 0“ 

= 


x(t) + 



.-1 0_ 


. 1_ 


( 2 . 1 ) 


which IS completely controllable, sampling rates with periods that are 
multiples of TT lead to uncontrollable discrete time systems , while ,all 
other sampling rates lead to controllable systems. Thus, plotting J* 
as a function of A would give something like Figure 2. 

Even for such a system, decreasing the sampling period will 
sometimes give improved results. It is true that 

J* ( continuous time) = lim inf J*(A), (2.2) 

A 

that IS, for any A > 0, there exists a A^^ < A such that J*(Aj^) £ J*(A) . 

A 

In particular, if n is a positive integer, then iJ*(— ) ^ J*(A) , since 
the control which optimzes the system for A is also available when 
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Not all sysieras >are as a:il -behaved as that of 'Figuse 2 . For 
'the scalar case, that as, the system xn which u e R and X e IR. , we 
can actually determine the interval of discretization h which minimizes 
J, If J*(Aj^) IS the minimal value of J for a given A = A^, then we 
can show that* 


9J*(A) ^ „ 

9A ^ 


(2.. 3) 


■Thus, the optimal value of A xs A . Since in this case, the ’feedback 

min 

matrix is a scalar, our online computation involves *only one multipli- 
cation. Thus, A = - where is the time needed for a single 
min M M 

multiplication. 

We discretize the .system 

x(t) -^axfct) t hu(t) , x{0) — .Xq„ C2.-4) 


With cost functional 


J = lim 
T 00 


i r 

^ Jo 


2 2 
{qx + ru )dt. 


(2.5) 


where q, r > 0, according to the solution of Levis [10], For a given A, 
holding u constant over each interval, J in (2.5) becomes 


N-1 


J = lim J ^ [(x(kA))^Q + 2x(kA)Mu(kA) + (u(kA))^R], 


N «5 k=0 


( 2 . 6 ) 


where Q, M and R are scalars as follows; 

.A 


= / 

•'o 


. . 2at 3_ / aA . aA 

Q = I e qdt = ^ (e + 1) (e - 1) 


(2.7) 
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M I ^ as , , , . bq , aA 2 

M = I bq e [I e ds]dt = — ~ (e - 1) 

'O 2 a 


= f bq [ r 
*'0 •'0 

R - f tr + q(b f e^*ds)^]dt 

Jn Jo 


'0 
= aA(' 


ra^+qb^, . qb^ , aA , aA 

3 — ^3 " 3) (e - 1) 

a 2a 


The state equation becomes 




where 


and 


, aA 
<j) = e , 


^A 

D = I 4>(t, 
•'0 


0)bdt = - (e®^- 1). 


Now in steady state, the Riccati equation becomes 

y\2 DM 2 2 DM 2 2^ 

0 = -K^[0 - + (<f) - — )^(R + D^K)K 

R R 

^ 2 -^ 2 '' 

- K(R + D K) + {Q - — HR + D K) , 

R 


which simplifies to 


0 = -D^K^ + { (4)^-1) R - 2DM4» + QD^)K + (QR - M^) . 


If 


B = (<|)^-1)R - 2DM(J) + QD^, 


( 2 . 8 ) 


(2.9) 


( 2 . 10 ) 


( 2 . 11 ) 


( 2 . 12 ) 


(2.13) 


(2.14) 


(2.15) 


then 



-16- 


K 


^ (^^-I)R - 2DM4> + QD^ + 4D^QR - 4M^D^ 

2D^ V -4D^ 


Now 


B ^ (d) -1)R _ Mi. . £ 

^2 ,2 ^ ~ D 2 

2^-. 


(d)+l)R _ bcrCd)-!) ^<j) ^ q(({)^-l) 

o 9 Vi 4a 

2b ((})-l) 2a - (4,-1) 
a 


(<J>+1) 

/ra2+qb^\ 

<4>-i) 

Uab^ 1 

(i+i) 

/ra^+qb^\ 

(i-i) 

Uab^ 1 

te the second tem 

(i+i)^ / 

0 ? 2 
ra +qb \ 

(0-1)2 ' 

2ab^ / 

f^(0"-i) 

(aA) (ra^^ 
^ 


(aA)^ + _ q(4>+l) (aA) (ra^+gb^) 

a^ a^b^ (i-l) 


2. 2 


% (i-l)" 
a 


2 2 


4a'^ 


(i-1)' 


a 


/ ra^+qb^ \ ^ ( (|)-H) ^ . . 2 . q(ra^+qb^) (d>+l) (aA! 

“1^1-2 / ,,2 + 2 " 2,2,. 

2ab ' (?"1) ^ ^ ^ fQ)-lj 


.^+qb^) 

(i-1) 


(2.16) 


(2.17) 
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q(ra +qb (<|>+1) (aA) q ((j)+l) ((j)-3) q 

^ 2 2 ^ 2^2 
2b a (({>-1) 4a 4a 

/ ra^+qb^ ^ _ q(ra^+qb^) (<|)+1) (aA) 

\ 2ab^ / 2a^b^(4>-l) 


Note that 


aA 



(l+aA+ 


(aA)^ 

2 ! 


aA 




(aA)^ 

3 * 


which IS 1 when aA = 0. Thus, 

A->0 ab^ ^ s}\ ab^ / 



which IS the continuous time solution derived in [1] . 
Now It IS well known that 

J*(A) = jx^(0)K. 


It IS straightforward to show (see Appendix A) that 


3A 


> 0 , 


and so, by (2.21) , 


3J* 

3A 


(A) > 0 


( 2 . 18 ) 


( 2 . 19 ) 


( 2 . 20 ) 


( 2 . 21 ) 


( 2 . 22 ) 


( 2 . 23 ) 
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for all A ^ 0. 

We have shown, then, that for the scalar system described in 
equations (2.4) and (2.5), the optimizing A is Since the feed- 

back control law is of the form 

u = -kx, (2.24) 

^min DUst the time a controlling conputer needs for a simple multi- 
plication, T,,. In other words, A = x . 

M M 



Chapter III 


The Dxscrete Time Output Regulator 


In order to optimize a cost functional over a class of control 
algorithms, as discussed in Chapter I, it is, of course, necessary to 
develop the appropriate set of algorithms. The most basic such control 
that comes to mind is the linear, full-state observation, sampled-data 
control developed by Levis [10] , His solution is to control the system 

x(t) = Ax(t) + Bu{t) ; xCt^) = Xq (3.1) 

in order to minimize the functional 


J = I (jc'Qx + u'Ru) dt 

by approximating (3.1) by a discrete- time system of the form 


(3.2) 


“k ■ 


With cost functional 


oo 

= (x’Qx + u'Ru + 2x'Mu ) 

1=0 ^ ^ ^ ^ ^ ^ 


(3.5) 


where u^ = u(iA) and x^ = x(iA) for a given A, and where the parameters 
$, D, Q, R and M are given as follows 


$ = 


AA 

e 


D = 



e Bdt 


(3.6) 

(3.7) 
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(3.8) 

(3.9) 

(3.10) 


Levis found that the optimal value for G is given by 

G'= r“^M' + (R + D'KD)“^D'K(f ~ Dr’’^M') (3.11) 

where 

K = lim IC (3.12) 

k ->00 ^ 

IS the steady-state solution of 

[# - DR”%'] + [Q - MR"^M’] (3.13) 

with bolindary condition 

= 0* (3.14) 

The cost in this case is 

Now if u £ and X £ iR ^ , and and are the times needed 

M A 

"*for a computer to perform an elementary multiplication and addition, 

respectively, then the minimum online time needed to perform the vector 

multiplication G*x^ is mnX + m(n-l)T^, which is approximately mnT„ 

i. M A M 

if (we are assuming, of course, that all of the elements of G 

are non-zero - as they would usually be) . Thus the minimum interval 

of discretization is A - itmT . 

min M 

Let us consider the controllable 2-dimensional system 
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x(t) 


-1 

e ' 

x(t) + 

'o" 

. 0 

1. 


_ 1 _ 


u(t) . 


(3.15) 


We might suspect that if e is sufficiently small, the behavior of this 
system would resemble that of 


i(t) 



"O' 

x(t) + u(t) 

_^ 1 _ 


(3.16) 


which IS xmcontrollable , but which can be stabilized. If we define 


x(t) 



(3.17) 


then in system (3.16) , will decay to zero regardless of input, while 
x^ will behave like the scalar system 

X 2 (t) = x^(t) + u(t) , (3.18) 

and, as was shown in Chapter II, a decreasing A will result in a mono- 
tonically decreasing cost associated with X 2 (t) . Thus, the optimal 
linear control for (3.16) will be of the form 


u = k X 2 (t) , (3.19) 

and the optimal A is A - A ^^^ . in the case of system (3.15) , for a 
given A, the optimal control will involve two multiplications; as we 
shall prove in Chapter V, however, a better result can be obtained by 
implementing the one-multiplication law of (3.19) , if £ is sufficiently 
small . 

Clearly, then, in some systems many of the optimal gains are 
small, and in higher-dimensional systems a great deal of computational 
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sxmplicxty may be obtained by setting these gains to zero. Thus, 
excellent performance can sometimes be obtained by feeding back only a 
subset of the states, and, given the simpler form of the controller, 
using a much smaller A in order to achieve performance superior to the 
"optimal**. With this motivation, we will develop an optimal constant 
discrete-time partial-state feedback control. 

The following results are parallel to the continuous -time work 
of Levine [9] . We include a complete derivation. The equations defining 
the solution are (3.42) -(3.44) , 

Given the system 

^ 3 . 20 ) 

= Ck^, (3.21) 

we wish to select f" in the feedback law 

\ (3.22) 

in order to minimize (3.5), which leads to a cost 

oo 

J(F) = j tr| ' [Q-2MFC + C'F'BFC](f>Q^|, (3.23) 

where 

- DFC. (3.24) 

We use the "trace” form in (3.23) to avoid developing a feedback law 
that depends on the initial state of the system; our solution will be 
optimal in an "average" sense ([6), [7], [9]). 
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We will find the following identity/ due to Kleinman [5] useful. 
If we have a function J(F) which satisfies the condition 

J(F + eAF) - J(F) = etr[K(F)AF] + o(e) , (3.25) 


then. 


3j(f) 

3f 


M' (F) . 


(3.26) 


We proceed as follows . for small e we have « 0 for n ^ 2 , so we may 
approximate 


J(F + eAF) « -7 S tr|[(|) - eDAFC] • 

^1=0 ^ ^ 

[Q-2M{F+eAP)C+C'F'KFC + eC'F'RAFC + eC’AF'RFC] t<J>Q-£DAFC] . 

(3.27) 


To first order, for 1 > 1, 

[<f)Q - eDAFC] - e|(})^^"^DAFC + <{)|^^"^dAfC(})q +...+ dAfC())q^"^|- 

(3.28) 

(The expression in (3.28) is the identity when 1 = 0.) 

We will call the expression on the right hand side - eA^. 

Now we rewrite (3.27) as 


J(F+eAF) 



£A^] ' [Q-2MFC+C'FKFC](j)Q^ 


+ e(|)^ {-2 mAfc + c'f'rAfcia, 




2M(F+AFG) C 


so. 


+ C'FRFC + eC'F'RAFC + eC'AF'RFC>, 


(3.29) 
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< x > 

J(P + eAF) - J(F) = J X) tr{-£A^[Q - 2MFC + C'F'RFClcj)^^ 

1=1 

+ e4>Q^ [-2MAFC + C'F’RAFC + C'AP'RFC](j)^^ 

- f 

- e<j)Q [Q - 2i5FC + C'F’RFC]A^} 

+ tr{e(-2MAFC + C'F'KAFC + C'Af'RFC)}]. (3.30) 

How, if we let 


Qq = Q - MFC - C'F'M' + C'F'RFC, (3.31) 

and use the trace property tr{xAFY} = tr{YxAF} to get each term to end 
in Af, we get 

J(F + eAF) - J(F) = j tr{-2CMAF + 2 CC'F'RAf} 

+ f tr {Y) -2CA ^ MAf + 2CA ^ 'cf'RAP] 

Z o u u u 

1=1 

oo 

- 2oE + C(j)Q4)Q^'QQ(f.^^”^DAF 


SO 


= e tr rc(j) ^(f) ^ C'F'rAF - C(f) ^ MAP] 

1=0 ^ ° ^ ^ 

^ X”1 I 


1=1 p =0 


(3.32) 


If" S 
'x?i I 5 


(3.33) 
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Now, let 


I' = IZ 


1=0 


CO 

K = 4>0^ 


1=0 


<j>oL<|)o’ = L “ I, and 


'J’o'^'^^o = - 2o- 


3 J 

Then if we set ^ - 0 in order to find the minimum for J, 
equation (3.33) becomes 


«> 1-1 


0 = RFCLC - M'LC -EL D' 'QQ(j)Q^((j)Q^) *C’ 


1=1 : i =0 


But 


oo oo 


1=1 j=0 j=0 i-]+l 


00 oo 

3=0 1=3+1 


If we let k = 1-3-1, then (3.39) becomes 


OO CO 


E E <C>'8oOo’<^o’'> - £ <C>-«oC^ £ *0 

3=0 k=0 k=0 3=0 


= Kcij^L. 


Then substituting in equation (3.38) gives 


0 = RFdiC - M'LC - D'K(J)qLC' 


(3.34) 

(3.35) 

(3.36) 

(3.37) 

(3.38) 

(3.39) 

(3.40) 


= RFCLC - M'LC' - D'K4»LC' + D'KDFCLC' 


(3.41) 
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so the solution is 

F = (R + D'KD)”^(M' + D'K(f))LC' (CLC')"^, '(3.42) 

where 

CO 

K = E <!>o^'QoC' (3.43) 

1=0 

and 

OO 

L = E (3.44) 

1=0 

where 

<j>Q = 4> - DFC (3.45) 

and 

Qq = Q — MFC - C'F'M' + C'F'RFC. (3.46) 

As we note in Appendix C, equations (3.42)-(3.46) reduce to the usual 
discrete- time solution [10] when C = I, 

Now, the optimal cost for the partial-state feedback controller 
is 

J* = j tr {Xj^’KXq}, (3.47) 

where K is as given in (3.37). We might choose to compare the full- and 
partial-state feedback laws by calculating $, D, Q, R, and M for a 
given A, solve for the optimal full-state feedback solution, re-calculate 
the parameter set for appropriately reduced values of A (reflecting the 
reduced confutation time) , and determine the optimal part-state solution 
for these values of A. 


,.a’ 


'’IBlLiry OF THE 
IS POOB 



Chapter IV 


The Discrete Optxmal Minimal-Order Compensator 

One technique used to stabilize a plant system for which only 
- part of the state can be observed is the construction of a dynamic 

compensator ([2], [12J , [13]), i.e., a system that estimates the unknown 
part of the plant state based on the entire past history of plant obser- 
vations. In some cases [2] , the plant is not output stabilizable, and 
use of a compensator is a necessity. If the system is output stabili- 
zable, though, we may choose either a simple output feedback controller 
or a more computationally complex compensator. We would like to know, 
then, when to use the "faster" output feedback or the "more accurate" 
dynamic compensation. 

First, however, we must develop a discrete-time version of the 
optimal compensator. Our work here parallels the continuous -time 
derivation of Blanvillam [2] . The results are summarized in equations 
{4.51)-(4.55) . 

Let us consider the system 

x{k+l) = Ax(k) + Bu(k) (4.1) 

with observation 

y(k) = Cx(k) (4.2) 

where the vectors and matrices are defined as 
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l "1 

X =1 


1 


2 ^ 


with £ ]R € JR ^ ^ , 


C ^ I 


A = 


11 


mxm 

mx 

A 

A, 

11 

12 

A^, 

A 

21 

22 . 

t. X 



H' 


22 




B = 


B. 


B, 


With B^ e'JR‘"*, B^ eJR 


*12 


(4.3) 


(4.4) 


(4.5) 


(4.6) 


In the above, we have assumed a canonical form for the system, insisting 
that the observation matrix is of a particular form, as in (4^4) • In 
fact, Blanvillain has shown, for continuous - time , that every linear 
system is indeed equivalent to a system of this form, if one eliminates 
redundant observations. The argument is easily extended to the discrete- 
time case. 

We would like to choose a cost f motional to minimize, using a 
linear feedback control. Unfortunately, as in the continuous -time case, 
using a cost of the form 
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2 (x(i)'Qx(i) +u(i)'Ru(i) - 2x(i) 'Mu(i)) (4.7) 

1=0 

would lead to a control law that depends on the initial state x(0) = x^, 
that IS, the optimal linear control would not be a strict feedback law. 

We use instead a cost that leads to a feedback law that is optimal over 
the average of a set of possible initial states; it is of the form 

CO 

J= (x(i) 'Qx(i) + u(i) 'Ru(i) - 2x(i) 'Mu(i) ) }. (4.8) 

1=0 

We will additionally treat x^ as a random variable with statistics 

E{xq} = 0 ; E{XqXq'} = (4.9) 

Now, It should be noted that the system (4.1) -(4.6) can be written 
as follows* 


x^(k+l) = A^^x^(k) + 

(4.10) 

X2(k+1) = A^^x^(k) + A^2X2(k) + B2u(k) 

(4.11) 

y(k) = x^(k) - 

(4.12) 

Since Xj^(k) can be observed exactly, and u(k) 
the input to be 

IS known, we may consider 

v(k) = A 2 j^x^(k) + B 2 U(k) 

(4.13) 

with observation 


y2(k) = Xj^(k+1) - A^j^x^(k) - B^^u(k) 
so the system (4.1) -(4.2) may be written as 

(4,14) 
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x^Oc+l) = + v{k) (4.15) 

72 (k) =A^2^2^^^’ (4.16) 

If we asstime the compensator xs of the form 

X2(k+1) = Fx 2 (k) + Lv(k) + H72(k) (4.17) 

then we choose [12] L = I, and 

^ ” ^22 ** ^12' (4.18) 

where H is a design matrix; it can be shown ([2], [12]) that the error 

A 

e = X 2 - X 2 IS described by 

e(k+l) = {A 22 *• = Fe(k)/ (4.19) 

and thus H may be adjusted [15] for arbitrary error dynamics (assuming 
observability of the original system) . 

Now one problem with the formulation (4.17) is that Y 2 (k) is a 
function of the future state x^(k+l) (see 4.14). To eliminate this, we 
instead use for our compensator system the variable 


z(k) = X 2 (k) - HXj^(k) , so (4.20) 

z(k+l) = Fz(k) + PXj^(k) + Du(k) , (4.21) 

where F is as in (4.15) , and 

P = FH - + A^j^, (4.22) 

D = - HBj^. (4.23) 


This IS in fact the minimal Luenberger observer. To optimize the 
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cost (4*8) over H, we will for the sake of computational simplicity 
reconsider this in terms of the augmented state 



(4.24) 


where 

eCk) = ^2^^^ " ^2^^^ ” ^^{k) - Hx^(k) - z(k) * (4.25) 

We Will assume at this point that a "separability" property 

A 

holds, x.e,, that if u{k) is to depend on snd x^Ck) , as in 

u(k) = Gx(k), (4.26) 


then the optimal values for G and H may be designed independently of 
each other. This means, then, that the optimal feedback G* is exactly 
the optimal feedback matrix for the system (4.1) when the full-state 
is observed. Blanvillain [2] justifies this assumption by showing that 
It leads to an optimal compensator design matrix H* that is independent 
of Q and R, two matrices on which G* e^licitly depends. We feel, 
however, that this argument is circular, and much prefer the reasoning 
of Miller [13], who shows that for any given H*, use of any other feed- 
back matrix will lead to an increase in cost. The proof is mechanical, 
and easily extends to the discrete-time case; we will dispense with it 
here. 

We may now e:^ress the input as 


u(k) = G*x(k) - G*Ne(k), 


(4.27) 
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where 


>N 5= 


tnx(mm) 

^ (n-m) x(n-m) 


(4.28) 


!The closed loop syistem may ,be ithen described 


s(k+l') = r3c.(k) = 


'(4.29) 


where 


r = 


■A + BG* -BG*N 
0 F 


The cost (4.8) is rewn-tten as 


(4.30) 


J(H> - E{^ s« (k)S2s(k)} 
'0 


in which 


Q 


Tq + G*' 
L-N'G*'E 


RG* + MG* + G**M' 
RG* - N'G*'M* 


-G*’RG*N - MG*N 
N’G*’RG*N 


']■ 


(4.31) 


(4.32) 


If we let Sq - e{sqSq'}, then the cost becomes 


J(H) = tr{£ 


(4.33) 


which suggests a Lyapunov equation; so if A is the solution of 


r-AT + - A, 


(4.34) 


then uhe cost becomes 


J(H) = tr(A5Q) . 


(4.35) 
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Prom this point, the derivation of conditions for an optimal H* 
IS fairly mechanical/ and we will present only the skeleton of the 
argijment* For a more detailed exposition, see Appendix B. 

Onr strategy will be to isolate the portions of (4.35) that 
depend explicitly on H. From equations (4.9) and (4.25) we can show 
that 



where 


= e{ 






+ 1^2 - 


12 


- ^l2“' 


14.37) 


when Sq is written 



Although both and are functions of H, it will be seen that 
does not contribute to the cost (4.35). 

As to the solution of Lyapunov equation (4.34) , that is. 


(4.38) 



A = 




(4.39) 


it IS trivial to show that is the usual Riccati solution K of (3.14) , 
given our earlier assumption of separability. This is rather imsur- 
prising in light of the fact that our assvimed feedback G* is dependent 
only on K and the system parameters (see (3.12)) . This result leads to 
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the equation for reducing to 

=■ (A* + G*'B')A^2^, 

and if we assume that G* is chosen for a stable plant, and H*, upon 
which F depends ((4.18)), is chosen to stabilize the error ((4.19)), 
then 

This enables us to rewrite the cost (4. ,35) as 

J(H) = tr(KZ^) + tr(A 22 EQ) , (4.42) 

where A 22 turns out to be 

N'G*'(B'KB. + R)G*N + ^ 22 ^ ~ ^2* (4.43) 

Since only the second term of (4.42) is dependent on H, we will minimize 
the added cost 

J(H) = tr(A^2\^- (4.44) 

If we write (4.43) as 

«0 + '■'^ 2 ^ ' \ 2 ' 

noting that is independent of H, then since A 22 is the solution of 


a Lyapunov equation, (4.44) becomes 
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As we xntend to ininxmze J according to 


9h 


= 0 , 


H* 


(4.47) 


we may again, as in Chapter III, make use of Kleinman's lemma {(3.6)- 
(3.7)) by getting an expression for J(H + eAH) - J(H) in the appropriate 
form. It then turns out that 


' ® i 2 ^ <^11 + 

where W is the ej^ression 


(4.48) 


W = S fSq(P^) 
1-0 


(4.49) 


which naturally appears in (4.46); by substituting H* into the Lyapunov 
equation associated with (4.49) r we get 

" ' \ 2™22 - * '‘ 22 ®'^ 2 ' '^1 + '^ 12 ™ 12 >‘^< h 2”''22 ^ 12 > 


^22 

Thus, the minimal increase in cost due to the compensator is 


(4.50) 


where 


J* = tr(A*2EQ*) , 


Eq = . 1^2 - 


‘*'22 - «*^ 2 > A ! 2<®22 - H 2 “*’> * ^22 


= H'K(B'KB + R)KN 


»* ' * ''22®i2> \2’"^2>'"' 


(4.51) 


(4.52) 


(4.53) 

(4.54) 
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and 


^ ^ 22^^22 ^22 " ^^12 ^ 22 ^^ 2 ^ ^^11 


-1 


. (A^2«S2 ^ ^12> 


(4.55) 


These equations, of course, may all be solved offline. However, 
the online construction of the state estimate x, needed to implement a 
feedback control, may be very time constiming. An examination of 
equations (4.20) - (4.23) show that (n-m) (n+m+1) multiplications will be 
needed for state reconstruction, in addition to n for feedback; only m 
will be needed for partial-state feedback. 



Chapter V 


An Example* Full- vs. Partial-State Feedback 

As we noted in Chapter III, we may sample outputs, and calculate 
op,timal controls, at a greater rate when we use a partial-state feedback 
algorithm than when we use full-state feedback. In other words, even 
if we are physically able to observe the full state of a system, we 
might feed back a function of only part of the state, hence simplifying 
computation. In such a case, we will lose knowledge of part of the 
state that would have been available at a slov;er rate; in return, how- 
ever, we gain more frequent knowledge of the part of the state that we 
do access. 

Kokotovic [8] describes a class of systems, called *' e-coupled" , 
which provide a n^mber of illustrative examples. A large system is 
said to be e-coupled if it splits into several independent subsystems 
when a scalar parameter e is zero. As Kokotovic maintains, the amount 
of computation for continuous-time control is greatly reduced if we 
calculate the optimal regulators for the decoupled sxi»systems rather 
than for the full system; moreover, the performance of the suboptimal 
control IS fairly near that of the optimal. Since we are in fact 
concerned with a sampled-control, however, the suboptimal control will 
lead to a shorter possible sampling period and perhaps a better result 
them in the full system control. 


- 37 - 
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In particular r if decoupling of a completely controllable system 
results in one subsystem that is uncontrollable but stable, and one that 
IS controllable, then it might be advantageous to feed back only the 
states that belong to the la*tter subsystem ♦ will consider a simple 

2-dimensional example of this. We look at the system 


x(t) 


'-1 

e 


"o’ 



x(t) + 


_ 0 

1^ 


_1_ 


(5.1) 


which IS controllable for all € ^ 0 ^ If e is very small, we might 
expect that (5.1) will act very much like 


x(t) 


-1 o' 


“o' 


x(t) + 


- 0 1_ 




(5.2) 


In fact, this new system (5.2) is output stabilizable; if we let 


x(t) 


"x^(t)“ 


(5.3) 


then Xj^(t) tends to zero regardless of the xnput, while the behavior of 
X 2 (t) IS described by the scalar equation 

x^Ct) = x(t) + u(t) , (5.4) 


which IS obviously controllable. 

Since only the cost associated with ^ 2 (t) is determined by the 
feedback law, the cost criterion, of the form 


r" 

"l 

o' 

J = [xMt) 

•'0 

^0 

1 _ 


x(t) + ru(t) ]dt. 


(5.5) 
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wxll be itiinimized by choosxng as fast a sampling rate as possible; it 
follows from Chapter II that J will be a monctonic function of the 
sampling interval A. What we will show is that if £ in system (5.1) 

IS sufficiently small, the sampling interval chosen in order to minimize 
a cost of the form (5.5) should also be as small as possible. 

We will assume for the sake of computational simplicity that 

2 

£ « 0. We define the matrices 


A = 


-1 

0 


e 

1 . 


(5.6) 


and 



(5.7) 


and using the usual series definition 


AT ^ ,2 ^ ^3 ^ ^4 ^ 

e = I + TA + — A + — A + 4 T A + . . . 


(5.8) 


we obtain 


AT 

e = 


1 0 
0 1 

0 




-1 £ 

0 1 


T 

21 


1 0 
0 1 


31 


"1 

0 


CsinhT 

T 


£ 

1 


+ ... 


(5.9) 


Now using the definitions of (3.7)~(3.11) / we have 


A aA 
f = e = 




esinhA 


(5.10) 
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EEPRODUCIBILITY Of 
QRIGBSfAL PAGE IS PQPt 


D = 


r ecoshA "1 
- 1 


Q = 


1-e 


-2A 


M = 


2 A+e~^^-l^ 


e ^2A+l-e~^^ y 


1 , A ,,2 
I -1) 


"9A 

e ^2A+e -1^ 


2A , 
e -1 


and 


R = (r+l)'A + 


Ce^-1) Ce^-3)i 


We shoul’d. note that the dxscrete" time system 


x(k+l) = $x(R) + Du(k) 


IS completely controllable » since 


$D = 


e(e ^coshA + e^sinhA - sinhA) 
, e (e -1) 


so 

detjfD = e (1-cosh (2A) ) , 

which IS nonzero when e 5^ 0 and A 5^ 0 . 

Now suppose we would like to determine the form 
cost if we feed back only part of the state / i.e.» 
tion and the one that follows we will make liberal use 


(5.11) 

(5.12) 


(5.13) 

05.14) 

(5.15) 

(5.16) 

(5.17) 

of the minimal 
In this deriva- 
of the approxima- 


tion 
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z _ z{a-eb) ^ za-ezb 

a+eb “ 2 ^2. 2 ~ 2 

a -e b a 

Looking at (3.23) , if we let 



(5.18) 


(5.19) 


then the expression D'KD becomes 

D'KD = 2ek2COshA(e^-l) tk^ (e^-1) ^ , 
and D'KO is the matrix 

D’K# = ^coshA + k2(l-e ^) 


(5.20) 

ek^ (e^^sinhA) + k2(e^^" e^)J • 

(5.21) 


Then, 


M' + D'K$ = 


(5.22) 


Now of course we can define 


C - [0 1] 


(5.23) 


for use in equation (3.23) . If we define 



.(5.24) 


then 
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LC ' (CLC ' ) 


= 


(5 


We can develop a property of L which will simplify things con 
siderably. We know from equation (3-23) (repeated here) that 


F = (R + D'*KD)~^(M' + D'R$)LC (CLC') 


.(5 


and in this case, F will be a scalar. Then 


f - DFC 


(5 


becomes 


^0 = 


-A 


e 

0 


,e(sinhA - FcoshA) 
e^ - F(e^ - 1) 


and using the definition of L in equation (3.15), 


( 5 . 


L= E (0(0'. 
1=0 ^ ^ 


(5. 


It IS easily determined that 
-2A , -4A 


= 1 + e 


+ e + . . . = 


1-e 


-2A ' 


(5. 


and that may be written as 


2 2 


(5. 


where I 2 a function of A; that is, we will be able to make the 


.25) 


.26) 


.27) 


28) 


29) 


30) 


31) 
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approximaticm 

^ 0. (5,32) 

Moreover^ we can derive a similar property for K, Recalling^ that 
the definition (3.16) is 


K == 


E 


i=0 


($ ’Q $ ^ 
'■^0 ^ 0 


(5.33) 


where 


Qq = Q - MFC - C'F'M' + C'F'BFC 


1-e 


-2A 


f [ 2 A+e"^^-l-F( 2 A+l-e"^^)] 


f [2A+e"^^-l-F(2A+l-e~^'*^) 1 

4 


2A 

^^ 2 . (e -1) „ , A , , 2 

RF +-i — ^ — - -F e -1) 


(5 . 34) 


we can readily see that 




(1 + e-2* + + 


) = 


1-e 


-2A 


2 d-e’^^) 


1 . 

2 


(5; 35) 


and 

' (5.36) 

where is not a function of e. 

Now we maintain that is the solution of the Riccati-like 
equation associated with discrete-time linear control of the scalar 


system 
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x(t) = x(t) + u(t) , y(t) = x(t). 


(5.37} 


with cost functional 


^2 ~ f + ru(t)^]dt. 


By setting b = q= a= lin equations (2.5) -(2.10) , the form of the 
Riccati-like equation (2.12) becomes 

0 = -(e^-1) V + [(e^^-1) ((rtl)A + j (e^-1) (e^-3)) 

-(e -1) e + -^(e +1) (e -1) ]k 

+ j (e^^-1) r(r+l)A + j (e^-1) (e^-3) ] - •- (e^-1)^, (5.39) 


0 = -(e^-l)k^ + [(r+l)A(e^+l) - 2(e^-l)]k + j (e^+1) {r+l)A+e^-l. 

(5.40) 

In the two-dimensional case, the equation describing k^ that results 
from the Lyapunov equation 




(5.41) 


k^Ie^ - F(e^-l)]^ « k^ - + |(l-e^^) + P(e^-l)^, (5.42) 

where R is as in (5.14}* 

Using the rules ek^ « 0, eZ^ « 0, and ^ 2^2 ^ make the 

approximations from (5.20) and (5.22), 


R + D'KD « (r+l)A + (e^-l)^^ + j(e^-l) (e^-3) , 


(5.43) 
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M' + D'K(f) W 

/2A+e“^+3 \ 
\ ^ / 

+ k^d-e"^) i k3e^(e^-l) + 





(5.44) 


and so, from (5.26), 


k3e^(e^-l) + j (e^-1)^ 
(r+l)A+(e^-l) |^j(e^-3) + (e^-l)k 3 j 


(5.45) 


Now direct substitution of (5.45) into (5.41) would give us a 


cubic expression in k^; we notice, however, that in the subexpression 


, „ 2 , A ,.2 , < n „2 „ 2 ‘,. , A -.2 . 

k^P (e -1) + RP = F (k 3 (e -1) + R) , 


(5.46) 


the coefficient of F'^ is just the denominator of F; in fact, (5.46) is 
just 


„„ A, A ,, 1 , A ,,2^ 

F(k 3 e (e -1) + “ (e -1) ) 


(5.47) 


Equation (5.42) is then just 


k^e^^ - 2k3P(e^-l)e^+ P(k3e^(e^-1) + |-(e^-l)^) 
' = k^ + |- (1-e^^) + F(e^-l)^ 


(5.48) 


which simplifies to 


, , 2A A, A 1 „2A. . 1 , A 1.2 

k 3 (e -1) - Fk^e (e -1) = j (1 - e ) + j (e -1) F. 


(5.49) 


If we divide by (e -1), and substitute (5.45) into (5.49), we have 


-k 3 (e^-l) + k 3 [(e^+l) (r+l)A + - (e^^-1) (e^-3) - j e^(e^-l)^] 

+ j (e^+1) (r+l)A + j (e^^-1) (e^-3) + | k 3 (e\l) (e^-1)^ 

1, A ,,3 1 . A, A 1.2 _ 

- - 1 ) - 2 ^ 3 ® - 



-46- 


= + k^rCe^+l) (r+l)A - 2(e^-l)] 

+ i (e^+D (r+l)A - 1 + = 0, 


(5.50) 


which IS exactly the expression in (5,40). Thus, if we let k(A) be the 
solution of (5.40), the matrix K becomes 




Sk2 

k(A) 


(5.51) 


and if the initial state is given as 


x(0) = 


_ 

01 


02 


(5.52) 


the resultant cost is 


.J^(A) 


- i fi 
“2^2 


*‘ol * 2 '* 2 * 01 -=' o 2 * ’"''>’‘02 


]■ 


(5.53) 


We now go to the case of full-state feedback control. Here, P*, 
the feedback matrix, will be 2-dimensional, e.g.. 


F* = [f3_ f^] . 


(5.54) 


we will also denote the steady-state solution of the discrete Riccati 
equation (3.14) as 


K* = 


ki* k^* 
k2* k3* 


(5.55) 


Unfortunately, xn thxs case we wxll not have any terms that are 
zero xn the closed loop transxtxon matrxx, unlxke the previous case of 



partial-state feedback. Thus, a direct attempt to solve {3.14) will 
result in three coupled non-linear _equations. We can, however, make 
the simplifying approximations 


« kj^ + ekj^ 
K 2 * « k^ + £^2 
K3* « k3 + ek3. 


(5.56) 

(5.57) 

(5.58) 


where k^, k^^, k^, k^, k 3 , k 3 , are independent of e, and need to be 

determined. These follow from the Taylor series expansions 

3k 2 92 j^ 

k (e) = k (0) + (0) + ~ i (0) + ... (5.59) 

1 1 3e 21 3^2 

and our assumption that Rs 0 for 3^2. Note that we are implicitly 
assuming that analytic in some region around e = 0 (see Com- 

ment 1 -at the end of this chapter) • 

"In fact/ since k = k (0) / we can easily see that ^ = 0 r 

11 12 2 

and k^ = k(A) * Then the resultant cost becomes 


Jf(A) 


2 L 2 




2®'^2*01='o2 


+ 0t(i) + 


“' s ’ *02 


]• 


(5.60) 


We can now compare costs (5.53) and (5.60). First we make the 
reasonable assumption that we have preselected a maximiun sampling 
period, that is, A is bo\mded above as well as below for both the full 
and partial-state feedback algorithms. Then, since the discrete time 
system (5.15) is controllable for all A 5^ o and e ^ 0, we know jk^^I , 
Ik^l, |k 3 | and Ik^l each have an upper bound over the acceptable range 
of A. Suppose, then, that the sampling rate for full-state feedback is 
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A* and that for part-state feedback xs Then the dxfference in the 

two costs IS 

J^(A*) - Jp( ^*) = 2’[^%^01 ^01^^02 ^ ^^3^02 

+ (k(A*) - k( ^*))Xq2] (5.61) 

Now, as we showed ui Chapter II, 

k(A*) - k( “A*) > 0; (5.62) 

we wxll defxne 

6 ^ k(A*) - k( |a*). (5.63) 


Thus, for any samplxng perxod A* and any boiinded set of xnxtxal states 

such that for some system of the form (5*1) we have 
2 


e(k 2(k -k ) — + k ) < 6, 

^ X 2 2 2 X 3 

02 


(5.64) 


we may conclude that 

,J^(A*) - Jp'( |a*) > 0. 


(5.64) 


In other words, there will be some system (characterized by e) for which 

(5.65) holds. Of course, E must be sufficiently small for our approxima- 
2 

txon e K 0 to hold. 


Comments 

1. Our assumption that K(e) is analytic is indeed 3 ustified. 
Results of this nature for the differential Riccati equation follow 
directly from a theorem of Pontryagin [14] (pp. 170-181; also see 
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Kokotovic [8] ) . We may extend these results as follows : consider the 

dxscrete-time equation (3.14) to be of the form K = f(K Then 

K - f (K) IS the steady state solution. Now consider the equation 

— rr — = f(K(t)) - K(t) . From Pon try agin* s conditions, each solution 
at 

K(t) of this equation is an analytic f\mction of e. But surely one 

/\ -A. 

solution is K(t) = K, that is, K(t) is constant. Thus, K is an analytic 
function of £. 


2. The conclusion of this chapter can be extended to the more 
general system 

i(t) = 

where b < 0. Then, the new $ matrix takes on the form 


b s 


0 


x(t) + 


0 a 


1 


u(t) , 


$(A) 


r bA 

e 

0 


ef (a,b,A) 

aA 

e 


/ 


where f(a,b,A) = — sinh(aA) , if b = -a, and f (a,b,A) = ® ® — , if 

a ij— a 

]b[ 5 ^ |a| . In either case, the reasoning of this chapter holds up 
in every detail. 


3. In fact, there are apparently n-dimensional analogues of 
this example. We will return to this in Chapter VI. Hie example of 
this chapter is also relevant to the dynamic compensator of Chapter IV, 
if we suppose that, for (5.1) , we are only able to observe X 2 (t) . In 
this case, partial-state feedback is still superior to compensation, as 
compensation could not give us even as good a result as the optimal 



full-state feedback law 



Chapter VI 


Conclusion 

The mam contribution of this thesis, in our opinion, has been to 
formally state how the fundamental characteristics of digital computation 
cause the saimpling inteirval A, as well as the usual feedback matrix, to 
be an optimization parameter* In this light, we have developed several 
on-line control algorithms, particularly a partial -state feedback law 
(Chapter III) , and an output compensator (Chapter IV) . We were able to 
explicitly state how, due to the question of relative computational 
complexity, one algorithm might be preferable to another; we derived an 
example (Chapter V) of a system where this was clearly illustrated, 
in order to 3ustify our general point of view. 

Since this work is intended mainly to provide a framework for 
further research, perhaps it is more appropriate to discuss the key 
questions that remain to be considered. In particular, we have not 
been able to devise a method for selecting the optimal sampling interval 
A » A* for a general class of systems, even for a particular control 
algorithm. One reason for this, as we have mentioned is the non-mono- 
tonicity of cost with respect to A. While this fact is disappointing, 
however,, it is not a fundamental obstacle to solution of the above 
problem. In many situations that occur in the theory of optimal control, 
It IS possible to optimize a control over two parameters; we do not 
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usually depend on the fact that the optxinal choxce for one parameter xs 
an extremum of xts permxssxble range* A far more serxous questxon xs 
that of "separabxlxty” • The technxque used to develop the optimal com- 
pensator (caiapter IV) , for example, relied on the fact that the observer 
and feedback parts could be designed independently ♦ In th^ case of A 
and F, however, xt is foolish to talk about separability; the optimal F 
fundamentally depends on our choice of A* 

The example of Chapter V, however, does suggest how we might 
extend this work to produce some useful results. Let us consider ]the 
general e-coupled system of Kokotovic 18] in the partioned form 



( 6 . 1 ) 


where and are and m^-dimensional substates^ n^ + 

and .are and x^-dimensional s.ubcontrols, r^ + = r. Then, 

the discrete vOptimal full-state feedback control (3.6) -(3.14) will 
anvoive rn online multiplications (and r(n-l) online additions, which 
we will ignore) . If we were merely to consider the systems 



X 


20 


^1^10 ®l“lO 


V20 


( 6 . 2 ) 

(6.3) 


we would ixnd that the optxinal control would xnvolve -only + -^ 2^2 

multxplxcatxons , whxch xs less than rn. We suggest that, usxng the 
assumptxons and technxques of Chapter V, that for any A, the dxfference 
between the costs resultxng from optxmally controllxng (6.1) or con- 
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trollxng (6.2) -(6 ,3) xs a Ixnear fxinctxon of e. Note, however, that 
unlxke the example of Chapter V, thxs does not demonstrate the super- 
xorxty of the snboptxmal control over the optxmal; the earlxer example 
depended on. the fact that the costs of the decoupled subsystems were 
monotonxc functions of A.. Thus, thxs approach wxll not be helpful xn 
controllxng (6.1} unless we know something of the behavior of (6.2) 
and (6.3), and are able to study system perfomance as a function of A. 

In conclusion, it is clear that the use of digital computers xn 
controlling large-scale systems will require the examination of issues 
such as those outlined in this thesis. It is hoped that our efforts 
will provide some useful insights for future researchers in this area. 



Appendix A 


We herein prove that 

i K (A ) , > 0 

for all A ^ 0, where K is given in equation (2.14) . 
If we define 


(A.l) 


G(4) = , ,j.2, 

2b a((j)-l)' 

we see immediately that 

G(A) > 0, {A. 3) 

for all a, and 

K{A) = aG(A) - •^ + yG(A)a(aG(A) - % . (A.4) 

a ^ a 

Moreover, since the scalar system (2.4) is obviously observable, the 
solution (A.4) must be positive definite (see [10]) and thus the radical 
in (A.4) must be real, i.e., 

G(A)a(aG(A) - J) > 0, all a. (A. 5) 

Now we calculate 


8K(A) 

8A 


aG*(A) 


j G'(A) [2a^G(A)-q] 
7a^G(A)^ - qG(A) 


This will have the same sign as 


(A.6) 
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F(A) == aG'(A) 7a^G(A)^-qG(A) + aG(A) 


{A. 7) 


where 


G’(A) = 


(ra^-t-qb^) {<j)^-2aA(l)-l) 
2b^a(^-l)^ 


(A. 8) 


and aG' (A) will have the same sign as 


L(aA) = (|) - 2aA4>- 1. 


(A.9) 


Then, we calculate 


= 2 ((}) - 5,n4» - 1) , 


^^^ = 2(1 - J ), (A.li: 

and note that = 0 at aA = 0 (cj) = 1) * Since — takes on the same 

9l 9l 9Ij 

sign as aA, — has a minimum at aA = 0, and thus — > 0/ and — > 0* 

- d<p - d(p 9 a ““ 

But since L(0) = 0^ L(aA) will take the same sign as aA. Then aG*(A) 
must have the same sign as aA, and G*(A) ^ 0* 

Now, suppose a 0 ; then 


'aV(A) - qG(A) + aG(A) - ^ 

4&a 


= -aG(A) “ q/a G(A) - 1 + 


2a^G(A)- 


If we denote 


X(A) = —2 , 

a G(A) 


then (A. 12) becomes 



-56- 


-aG(A) 



Clearly X(A) will be at a maximum for the value of A V o for 
IS minimal; since G* (A) >0, this point occurs at 


G(0) 


2 2 
= ra +qb 

2x,2 

a b 


t 


and thus 

,2 

X(A) _< — ^ ^ 1 for all a, all A ^ 0. 
ra^-Hqb^ 

Since G(A) ^ 0 for all aA, 

X(0) ^ 0, all a^ all A 0. 

o 

We can use the Taylor Series expansion 


= 1 - 


X X" 3X" 15X 

2 “ 4(21) ■ 8(30 ^ 16(40 


and note that, for all a and all A ^ 0, 

- 1 + 1 < 0 . 


Thus (A. 12) and (A. 14) are negative. Since for aA _< 0 
aG' (A) ^ 0, we can conclude that F(A) ^0, for a £ 0, and 


0 


3A - 
for a ^ 0. 

Wow suppose a > 0. Then (A. 12) becomes 


aG(A) ^ " f]’ 


(A. 14) 
which G(A) 

(A. 15) 

(A. 16) 
(A. 17) 
(A. 18) 
(A. 19) 

/ 

(A. 20) 


(A. 21) 
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X 1 

Since ^ ^"2 ^"2 ' ej^ression in (A. 21) is now positive. 


Since aG' (A) ^0, we may then conclude 


9k(A) 

3A 


> 0 


(A. 22) 


for all a, and all A > 0. 



Appendix B 


We fill in the gaps in the derivation of Chapter IV, 


B,1 Evaluation of A, 
If we define 



A 

A 

A = 

11 

12 


A' 

A* 


r 12 

22 J 


{B.l.D 


then the Lyapunov equation of equation (4.34) is equivalent to 
(A' + G*'B»}Aj^^(A + BG*) + Q + G*'RG* + MG* + G*'M' = 

(B.1.2) 

-(A* + G* 'B ' ) Aj^ j^BG*N + (A' + - G*'RG*N 


- MG*N = A^2 {B.1.3) 

K'G*'B'A^j^BG*N - F'A^^BG*U - N'G*'B'A^2F + F'A^^F 

+ N'G*'RG*N = A^^. {B.1.4) 

Now, comparison of (B.1.2) with equation (3.13) shows that A^^^ is 
indeed the steady-state solution of the full-state output regulator 
equation. We will denote A^^^ as K. Then from (3.11) we see that 

G* = r”^M’ - (R + B'KB)”^B'K[A - BR~\']. (B.1.5) 

Then, (B.1.3) may be rewritten, using (B.1.5) 
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^2 -(A'+G*'B')KBG*N + (A'+G*’B') - MR~^M’N 

- (A'-Mr“^B')KB(R+B'KB)~^R (R+B*KB)“^B'K{A-BR~^M')N 

- M(R+B'KB)“^B-'K(A-BR~^M’}N - {A’-MR~^B')KB{R+B'KB)~Hi'N 

+ mr”^m'n + M[R+B*KB]"^'K{A-Br“^M^)N, (B.1.6) 

which through judicious substitution of (B.1.5) and several cancellations 
simplifies to 

A^2 = (-A’+MR"^B’)KBG*N + (A'-Mr“^B’)KB(R+B'KB)“^’KBG*N 

+ (A+G*’B)A^2^ + (A*-Mr“^B*)KB(R+B'KB)"^RG*N, (B.1.7) 

and if we recognize the pattern 

(A'-MR~^B')K(-B+B(RfB'KB)”^*KB - B {R+B 'KB) "^R) G*N 
= (A ' -Mr” ^ ' ) K (-B+B (R+B 'KB) (B 'KB+R) ) G*N 

= 0, (B.1.8) 

then (B.1.7) becomes 

Ai2 - (A'+G*'B’) A^2^. (B.1.9) 

If we assume that H* is restricted to values for which the error e 
(4.16) IS stable, then 1 If| | < 1; if G* is chosen for stability of the 
plant, then ||A'+G*'B'j| < Ij thus, 

11^211 - I I ^ 11^211' (B.1.10) 

unless [jAj^ 2 ll therefore, [IAj^^II “ 

Equation (B.1.4) may now be simplified to 
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N'G*’ (B'KB+R)G*N + ~ ^ 22 * 


(B.1.11) 


givxng us 


A = 


K 

0 



SO in fact the total cost is 


tr<KS:„) + tr(A22E„) . 


( 8 . 1 . 12 ) 

i •K i 


(B.1.13) 


B.2 Choice of H*. 

Clearly^ we want to choose H to ^nimxze idle added cost 
J = tr(A 22 ®Q). We set 


3 h- 


H* 


0 , 


(B.2.1) 


and noting that ^^2 is the solution of Lyapunov equation .(B.1.11) , we 
see 


00 

F^'n'G*' (B''KB+R)G*NP^. (B.2. 2) 

1=0 


If we let 

= N'G*' (B’BCB+R)G*N, (B.2. 3) 

then we may rewrite tr(A 22 EQ> as' 

00 

J(H) = tr{tdQ Z) F(H)^Eq(H)F(H)^'}. (B.2. 4) 

^ 1=0 


Again, as in Chapter III, we will make use of Kleinman's lemma summarized 
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in (3.25) -(3.26) to determine 

Eq(H+£AH) = Eq(H) + '■'^12^ 

P(H+eAH) = P(H>-eAHAj^ 2 * 

Since 

00 

J(H+eAh) == tr{MQ Y. (P-eAHA^^)^ 

1=0 

+ e(E^^H'-S^2) 'AH*)(F-eAHA^2^^'^' 

using the approximation 

1-1 

(f-sAha,_)^ w p^ - e y\ F^~ ^AHA-.F^ 

LZ ±Z 

3=0 

when 1 7 ^ 0 (and is the identity when i = 0) , we may rewrite 


(B.2.5) 

(B.2.6) 


(B.2.7) 

(B.2.8) 
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x’ 


x=0 

CO 1^1 


- £M X) Z) 'a* AH'P^') 'E F^' 

1=1 3=0 

+ p ^ Eq ( f ^^“^~^^ Aha ^ 2 ^^) ']) 

00 

= etr 2(S^,H«-Z )f^'m f^Ah 

1=0 ^ 

00 1“1 

- Y, Y Ah]>. 

1=1 3=0 


(B.2.11) 


We may now apply the Kleinman lemma to get 

00 

^ J ( H ) = 2 E f ‘ ■ 


x =0 


- Y Y [F^ *E 'EqF^MF^’]AJ^2 

1=1 3=0 

oo 

= 2 E . 


00 1-1 


3 M r>^T? la I 

12 


~ 2 Y S [F^ M fSqP'* * -" ]A 

i=L 3=0 


(B.2.12) 


Now we can make the manipulation 

Y Y [F^’MqF"-EqF^^"^"^^’] = i; f; ' 

1=1 3=0 3=0 1 = 3+1 

-T, t = <E f ’' m . f ’ hE f ’'*V’''> 

3=0 k=0 3=0 ^ k=0 


(B.2.13) 
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If we let 


OO 

E k k * 
F EqF 


k=0 


(B.2 


SO that 

FWF* = W - E-, 


(B.2 


and set J(H*) =0, (4.63) becomes 
uH 


0 = (2 X] fW)I(Z..H*'-E._) ' -FWA' ], 


(B.2 


SO in fact 




(B.2 


Solving this for H*, we see 


»* ■= <^i2 ''22”'^i2> '^11 + ®12®''i2>'''- 


(B.2 


We may then rewrite (B.2. 18)/ using (B.2. 15) , to obtain 


® - "** 12’"'22 * ' '' 22 ® i 2 ’"' 


+ - £^2 ^i2«*' 


= - ■“<^2'®^22 * ^12> ^ H* 


- <*22’«'l2 ^l2>«*' ^22 


(B.2 


SO 


^ ~ *22^^22 


^^12 ■*■ * 22 '^ H 2 ^ ^^11 ^ 12 ^^ 12 ^ ^^ 12^*22 ^ 12 ^ 


+ s. 


22 


.14) 


.15) 


.16) 


.17) 


.18) 


.19) 


(B.2. 20) 
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and we get the results of equations (4.51)-(4.55) . 



Appendix C 

Reduction of Results of Chapter III to the Full-State Solution 


Here, we let C I in equations (3^42) -(3*46) and show the 
results agree exactly with the work of Levis [10] . 

Equation (3*42) becomes 

F = (R + D'KD)~^(M' + D‘K(f>) 
which implies 

(R + D'KD)F = M' + D'K<J> 

= M’ + D'KDR~^M' + D'K(|) - D'KDR'"^' 

= (R + D'KD)r“^M' + D’K(<j) - DR"^M'), 
so 

F = R^^M* + (R + D'KD)'’^D'K(4> - DR~^M')- 
Moreover/ equation (3-37) becomes 

K = + Qq = ((j) - DF) *K(4) - DP) + Qq 

= (^ - DR“^M') 'K(<j) - Dr“^M') 

-(((})- DR~^M') *KD(R + D’KD)“^D')K(<f) - Dr‘'^M') 

-(<})- DR~^M') 'K{D(R + D'KD)~\)'K(4) - DR^^M' ) ) + Qq 
+ ({(j) - DR“^M')'KD(R + D'KD)“^D*KD(R + D'KD) “^D'K(<{) - DR 
= Qq + (4> - ' [K - 2KD(R + D'KD)"^D'K 

+ KD(R + D’KD)“S'KD(R + D'KD)“^D'K] (<p - DR‘^M') 


(C.l) 


(C.2) 

(C.3) 


•M')) 


(C.4) 
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Now Q. IS 

= Q - MF - F'M' + F'RF 

0 

= Q - MR~^M' - M{R + D’KD)”^D'K((j) - DR~^M'} 

- (<j) - DR^^M') 'KD(R + D’KD)”^M' - Mr"^M' 

+ mr"^m' + m(r + d'kd)~^d'k(^ - dr“^m‘)' 

+ ((})- DR“^M') 'KD(R + D'KD)“^M' 

+ ((J) - DR”^M')'KD(R + D'KD)“^R(R + D'KD) “^D'KCi}) - DR**^M'} 

= Q - + (i|) - DR"^M')’KD{R + D”B1))''^R(R + D"KD)~^D’K 

• ((j) - DR-^M'), (C.5) 

Now, to combine with the third term in brackets in (C.4) , we note 

KD(R + D'KD)~^R(R + D'KD)~^D*K 

+ KD(R + D'KD)'‘^D'KD(R + D*KD)‘*^D’K 
= KD(R + D'KD)“^(R + D'KD) {R + D'KD)“^D’K 

= KD(R + D'KD)“^D'K (C.6) 

whxch we may combine with the remainder of (C*4) to show 

K = ((J) - DR"^M’) • [K - KD(R + D'KD)‘*^D’K] - Dr“^M') 

+ (Q - MR~^M’) . (C.7) 

Equations (C.3) and (C.7) agree exactly with the results of 


Levis . 
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